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© In a partial pattern matching system, a dictionary pattern, 
apattern to be recognized and a binary-coding unit for giving 
a binary-coded image pattern by comparing each pwelwrth 
a predetermined voltage (10, 101 to 103) are provided. A 
polygonal approximation unit (104) gives a coordinate string 
of vertices of a polygon composed of e plural ity of successive 
straight line segments approximately representing e contour 
of the binary-coded image pattern. A contour segmentation 
unit (1 06) judges a comer point among the vertex coordinates 
of the polygon based upon a criterion dependent upon the 
shape of the polygon, the comer point being used for dividing 
the polygon into a plurality of partial pattern. In a matching 
unit (107. 109). a correspondence pair is given by subjecting 
each combination of the partial patterns of the dictionary and 
recognition-object patterns to matching comparison, and 
each evaluation of matching degree is given for each of the 
correspondence pair. 
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PARTIAL PATTERN MATCHING METHOD AND APPARATUS ' 



1 BACKGROUND OF THE INVENTION 
FIELD OF THE INVENTION 

The present invention relates to parts recog- 
nition by image processing, and more particularly it 
5 relates to a partial pattern matching method suitable 
for recognizing incomplete parts patterns including 
therein such as superposed or broken-out portions. 

in the Articles, Robert C. Bolles, "ROBUST 
FEATURE MATCHING THROUGH MAXIMAL CLIQUES", Proc. SPIE 
10 Technical Symposium on Imaging and Assembly, April, 1979; 
and Shigeyuki Sakane, "Segment Matching of Shapes Based 
on Clique Detection of Graphs" Electrotechnical Laboratory, 
PEL 82-84, February 1983, there is disclosed a pattern 
matching system in which a set of characteristics 
15 existing locally between a pattern to be recognized and 
a dictionary pattern are compared with one another. 

With conventional parts recognition methods, 
if plural parts are superposed one upon another or if a 
portion of the part is broken away, an entire agreement 
20 of the recognized pattern shape with the part shape is 
not obtained so that characteristic values are not 
correctly sought. Therefore, a correct recognition 
could not be attained, e.g., failing in recognizing the 
superposed parts, or overlooking the partial difference 
25 of such parts as having a broken out portion. 
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1 SUMMARY OF THE INVENTION 

It is an object of the present invention to 
provide a parts recognition method capable of attaining 
a high recognition rate with a low cost, which method 
5 is adapted to enable to recognize an incomplete pattern 
caused by superposition, breakage and the like of the 
parts thereby reducing the environmental conditions 
required for the recognition. 

It is another object of the present invention 

10 to divide or segment, for the purpose of pattern 

recognition, a contour line of a two-dimensional image 
into partial or local patterns and to give the most 
proper indexes representative of the matching state of 
an incomplete contour. 

15 It is still another object of the present 

invention to provide a method and apparatus in which a 
pattern recognition rate for the non-isolated or 
scattered object can be improved and the recognition 
can be achieved quickly. 

20 In the preferred embodiment of the present 

invention, since an incomplete pattern coincides in part 
with a dictionary pattern, in recognizing the incomplete 
pattern, the pattern is segmented into local patterns 
so as to match the local pattern shape and its position. 

25 In this case, in order to check the correspondence 

between the dictionary pattern and the local pattern of 
the pattern to be recognized, both dictionary and local 
patterns are divided into plural patterns in accordance 
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1 with the same criterion not depending upon the position 
and orientation of the patterns but depending upon only 
the contour shapes. 



BRIEF DESCRIPTION OP THE DRAWINGS 
5 Fig. 1 shows one example of automatic assembly 

systems to which the method and apparatus of the present 
invention is applicable. Fig. 2 is a schematic view 
showing the construction of an image processing unit. 
Figs. 3a and 3b are views showing examples of a 
10 dictionary pattern and parts pattern to be recognized, 
respectively. Fig. 4a is a sechematic view showing the 
construction of a parts recognition unit, Fig. 4b is a 
parts recognition flow chart according to the present 
invention. Fig. 4c is a view showing binary-coding, 
15 Figs. 4d and 4e are views showing the processes of 

polygonal approximation of contour lines. Fig. 5a is a 
view showing the polygonal approximation and a segmen- 
tation of the polygon, Figs. 5b to 5f are views illus- 
trating the polygonal approximation , Fig. 6 is a detailed 
20 illustration of the local pattern matching process of 
Fig. 4a, Fig. 7 shows schematic illustrations for 
explaining the local pattern matching, and Fig. 8 shows 
schematic illustrations of the evaluation for the match- 
ing results of each local pattern. 



25 DESCRIPTION OF THE PREFERRED EMBODIMENTS 

The embodiment of the present invention will 
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1 be described with reference to the accompanying drawings. 
Pig. 1 shows an automatic assembly station where a part 
supplied from a rotary parts feeder 1 is assembled on a 
chassis passing along a belt conveyor 2. The image of 
5 parts on the rotary parts feeder 1 is taken with a TV 
camera and input to an image processing unit 4. In 
the image processing unit 4, the presence or absence of 
a part to be assembled is recognized from the input 
image. If the part is present, the position and orien- 

10 tation of the part are calculated and the calculated 
results are transferred to an assembly robot 5. The 
assembly robot 5 moves its arm 5a in response to the 
information supplied from the image processing unit 4 
to grasp the part and assemble it. The rotary parts 

15 feeder 1 feeds the parts piled up on a center disk la 
to a peripheral truck lb* The parts on the truck take 
arbitrary and two dimensional position and orientation 
so that the parts may contact one another or may be 
superposed in part one upon another. Thus, the image 

20 of the parts occasionally presents an incomplete pattern. 

The arrangement of the image processing unit 
4 of Pig. 1 is shown in Fig. 2. An example of a 
dictionary pattern which is obtained in such a manner 
that a part to be used is photographed with the camera 

25 3 and the photographed image is converted into a binary 
code at an A/D converter of an image input board 10a , 
is shown in Fig. 3a. An example of a pattern to be 
recognized in which two parts are superposed one upon 
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* 

1 tfee other is shown in Pig. 3b. The apparatus and 
process for the recognition are shown in Pigs. 4a and 
4b, respectively. First, the image is input to an 
analog-to-digital converter 10 and converted into digital 
5 values each representative of the brightness of each 
pixel in the image (step SI) . Based upon the comparison 
results of the digital value at a comparator 102 with a 
binary-coding threshold from a register 101, a binary 
image is obtained, in which the contour line of the 
10 pattern to be yecogniaed represents to the boundary 
of black region and white region (step S2) . Black- 
colored small squares in Pig. 4c represent contour 
line constituting pixels obtained through binary-coding. 

The contour line or curve is approximated to 
15 a polygon made of several straight lines. First, 

the center of one of the contour points shown in Fig. 
4d is considered as a start point 201. The start 
point is registered as a vertex of a first approximated 
line segment. By introducing an imaginary circle 
20 having a radius e and a center P i for the next contour 
point, an upper tangent angle TA2 and lower tangent 
angle TAl relative to the start point are determined. 
In a similar manner, by introducing circles having 
radii e and centers of consecutive contour points, 
25 corresponding upper and lower tangents are determined. 
Among the obtained tangent angles , the maximum value 
of TAl is represented by MAX (TAl) while the minimum 
value of TA2 is represented by MIN(TA2) . 
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1 In the case that MAX(TAl) > MIN(TA2) is estab^ 

llshed, it is decided that the approximation for the 
first line is no more possible. Then, the contour point 
at that time is considered as an end point of the first 
5 line and is registered. 

Candidate points for the end point are judged 
by assuring the fact that after the renewal of MAX(TAl) 
and max(TA2) obtained from the tangent lines, a line 
CA drawn passing, through successive points is positioned 
10 between MAX(TAl) and MAX (TA2) . If the end point is not 
for the last data point, then using the point as a start 
point for approximation of another line, processings 
for approximating the other line are performed. For 
example, two parallel lines with a constant width 210 
15 may be used to determine a line ending at the vertex 

of the polygon. 

With such processings, a binary-coded image 
shown in Fig. 4c is transformed into a polygonally 
approximated contour made of a plurality of lines 207 
20 to 209 connected to each other as shown at the right 
of Fig. 4e. in a polygonal approximation unit 104 
shown in Fig. 4a, the pattern contour 211 shown in 
Fig. 4c is approximated to a polygon. A set of coordi- 
nates (x i# y i ) of the vertexes of the polygon are stored 
25 in a memory provided in the unit 104 (step S3). It 
is here noted that units 106 to 108 shown in Fig. 4a 
and described later include not shown memories, respec- 
tively. The vertex coordinates are given to a segmentation 
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1 unit 106 wherein the polygon made of plural line 

segments is segmented in association with several charac- 
teristic points of the polygon (step S4) . Pig. 5a shows 
an example of a polygonal approximation for the dictionary 
5 pattern of Pig. 3a. As seen from the figure, among 

angles between two adjacent sides , the angle (for example, 
6^ between the two sides on a smooth curve is small, 
while the angle (for example, 6 2 ) between the two sides 
on a corner of the pattern is large. By setting a suita- 
10 ble threshold in the succeeding segmentation step, it 
is possible to detect corners a, b, c, d,. e, f, and g.. 
The coordinates of these corner points with flags set 
and the remaining coordinates of the connection points 
between line segments can be output to a matching device 
15 107 (step S4). 

The pattern as shown in Fig. 5a can definitely 
be recognized. However, the recognition of an indeter- 
minate projecting shape 122 of a pattern 121 shown in 
a circle of Fig. 5b is unstable due to the coarseness 
20 of pixels and displacement of the meshed pixels from 

those of the pattern 121: in that the polygonal approx- 
imation and segmentation may be performed to have either 
line segments 123 with a single corner, or line 
segments 124 with two corners, or line segments 125 
25 with three corners. The more detailed description for 
this is given with reference to Figs. 5c and 5d. In 
the figures, x marks represent the centers of pixels. 
The two thick lines connected at a point h correspond^ 
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1 to two line segments 127 and 128 after subjected to 
the polygonal approximation. The circle about the 
point h indicates that the point h has been judged as 
a comer or vertex. 
5 In Pig. 5d, since the meshed pixels are some- 

what displaced as compared with those in Fig. 5c r the 
point 131 is positioned outside of two one-dot chain 
lines 140 and 141. In this case, another point i 
different from the point h in Pig. 5c is judged as a 

10 corner. Therefore , the rightmost point j on a line 
segment 138 will be recognized as a second comer. 

In order to obviate the unstable recognition 
due to the mesh displacement described above, it is 
possible to adopt a judgement criterion as shown with 

15 Pigs. 5e and 5f together with the judgement criterion 
using the foregoing angle threshold 8^. In particular, 
an angle cu is calculated by drawing broken lines 151 
and 152 passing through points k - n, k + n, and k, 
the points k - n and k + n being respectively apart 

20 from the point k by a certain number n (= 5, for example) 
of line segments. A similar calculation for an angle 
<* i+1 is performed between the point (k + 1) and points 
(k - n + 1) and (k+n + 1). If any one of the angles 
is judged to have a smaller value than the preset 

25 threshold a Q , then the corresponding point is recognized 
as a vertex and the other points can be excluded . 
Therefore, such segmentation as the lines 137 to 139 
shown in Pig. 5d is not brought about, but the point k 
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1 apart from the pixel point h of Fig. 5c by a single 
pixel is judged as a vertex. Thus, a polygon similar 
in shape to that in Fig* 5c can be obtained. 

Another vertex determination method will 
5 be described with reference to Fig. 5f . A certain 
number of contour line segments are chosen, and an 
arrow 162 is added. In the figure, a resultant vector 
162 of 10 vectors 161 is drawn on the middle vector in 
the vector train. Similar resultant vectors for respec- 
10 tive successive pixels are calculated. If the direction 
of the resultant vector has a large angle than a preset 
angle, then the corresponding pixel point is recognized 
as a vertex in the polygonal approximation. Thus, 
the tins table recognition as shown in Fig. 5d is avoided. 
15 The processes up to step S4 for the segmentation 

' of polygon described above with particular reference 
to Figs. 1 to 5f are performed first for the dictionary 
pattern and then for the pattern to be recognized in 
the practical field of pattern recognition applications. 
20 The pattern recognition (step S5) carried out 

with the matching unit 107 and an evaluation unit 108 
will be described in detail. Although the dictionary 
pattern and the pattern to be recognized respectively 
shown in Figs. 3a and 3b have in general different 
25 shapes, those patterns include the same portions in 

part. Therefore, if there exist many and the same local 
patterns as those in the dictionary pattern at the 
positions not contradictory to the positions in the 
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1 dictionary pattern, the pattern to be recognized is 
judged as inclusive of the dictionary pattern. 

The processes for the above judgement will 
be described with reference to Pig. 6. First, all of 

5 the combination of local patterns in the dictionary 
pattern and the pattern to be recognized are checked 
(step S51) . Next, among the local patterns judged as 
having the same shape, every combination not contradictory 
to the positions in the dictionary pattern is sought 
10 (step S52). The similarity of both patterns is judged 
based whether or not the sought combinations occupy 
a substantially large portion of the dictionary pattern 
(step S53) . 

At first, step S51 in Fig. 6 will be described 
15 in detail with reference to Fig. 7. The distance between 
the start and end points of a local pattern and the sum 
of lengths of each line of the local pattern are used 
as auxiliary characteristic values. For each local 
pattern in the pattern to be recognized, those local 
20 patterns in the dictionary pattern which have the values 
d and I close to the values of the local pattern in the 
recognition pattern, are searched. For instance, B 
and G is selected for B 1 . Thereafter, B f is superposed 
upon B or G by coordinate transformation in order to 
25 check the coincidence between both local patterns. That 
is, in other words, the start and end points of both 
patterns are positioned as close as possible to each 
other, then all of the shortest distances P. from the 
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1 vertiges of q^q pattern to the sides of the other pattern 
(for example, the length P^^ of a perpendicular from a 
vertex of B 1 to the adjacent side of B) is checked 
whether there is a longer distance than a preset threshold 

5 Dl. If it is decided to be coincident, coordinate 
transformation factors representative of a shift and 
rotation at that time are stored as a coordinate trans- 
formation matrix ir. The adjacent side corresponding to 
vertex may be plural in number, thus the coincidence 

10 judgement may be made when the sum ZP. of all of the 

i 

lengths P^ between the vertices and sides becomes 
smaller than a preset threshold D2. 

Step S52 in Fig. 6 will be described with 
reference to Fig. 8. The local pattern correspondence 
15 between the patterns decided at step S51 as having the 
same shape is shown in Table (a) . Th§ coordinate 
transformation matrixes it., (j « 1, 2, . ..) shown as a 
mother group in Fig* 8 for all of the above decided 
patterns, are divided into group 1 and froup 2 respec- 
20 tively having almost the same coordinate tjrajis formation 
matrix value, by using a criterion judging whether 
the matrix value is within a permissible range QV not. 
As a result, among the dictionary and the patterns to 
be recognized judged as having the same shape? Qnly 
25 those having not contradictory positional relation© tQ 
each other as shown in Tables (b) and (c) , i.e, group© 
of correspondence pairs of the local patterns c^n be 
obtained. 
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Table (a) 



A' 




A 


B* 


— 


B, G 


C 


- 


None 


D' 


- 


None 


E» 


- 


A 


F* 


- 


B, 6 


G' 




C, P 


H» 




D 


I" 




E 


J» 




None 


K» 




None 


L' 




B, G 





1 At step S53, the groups of the local patterns 

thus obtained are evaluated as to the degree of similar- 
ity. 

The evaluation employs the following evaluation 
5 function MATCH: 

MATCH = Z Wj ' f (j) 

j: the local pattern number of the dictionary 
pattern 

W^.: weight of each local pattern of the dictionary 
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pattern I Wj = 1 
j 



"l": indicative of the presence of the 
corresponding local pattern in the 
f (j) = ( object pattern to be recognized 

"D": indicative of the absence of such 
local pattern 



1 The weight is previously determined depend- 

ing upon the importance of the characteristics of the 
local pattern. By setting a threshold for the value 
of the evaluation function, the groups of local patterns 
5 having the value of the evaluation function MATCH 

exceeding the threshold are judged as the same as the 
dictionary pattern shown in Fig. 3a for example. 

Each local pattern belonging to the group 
which is judged having the same pattern as the die- 
10 tionary pattern, is part of the dictionary pattern not 
contradictory to the position and orientation thereof. 
The calculated results of the position and orientation 
of the part pattern in the pattern to be recognized are 
sent to the robot. Although it is not necessarily the 
15 case that the position and orientation of the part 

pattern can be calculated from the position and orienta- 
tion of some local patterns, it is possible to obtain 
the position and orientation of the part pattern by all 
means if suitable threshold values of the evaluation 
20 function and weight W. are set. 
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1 The present invention is not intended to ^@ r • 

limited to the recognition of assembly parts for an ! 
assembly robot , but other pattern recognitions are also 
applicable. For example r the present invention is 

5 applicable to a character recognition in which each 

i 

stroke in a thin character is recognized as a local 

pattern and the above evaluation function is used. 

Another application is to such a case as an additional 

part is mounted on a movable part on a mechanical base 
10 plate wherein the contour of the movable part is 

indefinitely recognized because of the unstable condition j 

of the background of the base plate. 

According to the above embodiments/ it is 

effective in that the recognition rate and speed for 
15 generally incomplete parts can be improved. j 

i 
t 

! 



I 
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1 CLAIMS i 

1. A partial pattern matching method for recogniz- 
ing an object pattern by comparing a dictionary pattern 
with the recognition-object pattern comprising the 
steps of: 

5 (a) approximating (Si, S2, S3) a contour line 

of a region in a two dimensional image for the recognition- 
object pattern to a pattern having a plurality of 
• consecutive line segments basing upon a criterion depend- 
ing only upon the shape of said contour line; 
10 (b) contour segmenting (S4) said polygon 

pattern basing upon the shape of said polygon pattern 
into a partial pattern having at least one polygon 
edge; and 

(c) matching (S5) said partial patterns of 
15 said dictionary pattern and recognition-object patterns. 

2. A method according to claim l f wherein said 
approximation step (a) includes a step of giving the 
coordinates of each vertex of a polygon approximated 
to said contour line, and said step (c) matches said 

20 local patterns by weighting each local pattern in 

accordance with the contribution to the characteristics 
of the shape of said dictionary pattern. 

3. A method according to claim 1, wherein said 
contour segmenting step (b) uses as a division vertex 

25 of said partial pattern a polygonal vertex where the 
angle between* the directions of one of said plurality 
of successive line segments and an adjacent line segment 
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1 changes to a large degree* 

4. A method according to claim 1, wherein each 
of said partial patterns of said dictionary pattern 
and recognition-object pattern has a simple shape as 

5 a whole without an inflection point. 

5. A method according to claim 4, wherein said 
matching step (c) includes the steps of: setting as a 
first characteristic amount the distance between both 
ends of each of said partial pattern and the sum of line 

10 segments on each of said partial pattern; selecting the 
partial pattern having said first characteristic amount 
within a preset error range and setting as a second 
characteristic amount the position and orientation of 
each selected partial pattern; and "setting as a matrix 

15 the amount of shift and rotation for transforming the 
coordinates of one of the partial pattern to the other 
of the partial pattern. 

6. A method according to claim 5 further com- 
prising: 

20 a step of classifying said partial pattern 

having the value of said matrix within a preset error 

range into the same group; 

a step of giving an index indicative of the 

presence or absence, within said same group, of the 
25 partial pattern of said recognition-object pattern 

corresponding to the partial pattern of said dictionary 

pattern; 

a step of setting for each partial pattern of 



" 3 ~ 

1 said dictionary pattern a weight dependent upon the 

characteristics of the contour of said each partial 

pattern; and 

a step of giving an evaluation function wherein 
5 the multiplication of said index by said weight is 

added for all of the combinations of partial patterns 

within said same group* 

7. A partial pattern matching method for recogniz- 

ing a recognition-object pattern by comparison with a 

10 dictionary pattern comprising the steps of: 

(a) binary-coding a two dimensional image 
obtained by photographing an object relative to the 
plurality of minute digital pixels and giving an image 
data representative of a contour line of said image; 

15 (b) approximating the image data representative 

of said contour line to a polygon having a plurality 
of successive line segments basing upon a criterion 
determining opposite ends of a line segment in accordance 
with a criterion that successive pixels are inside of 

20 two parallel going lines set in the direction of the 

successive pixels at a predetermined space therebetween, 
and giving a string of coordinates representative of 
respective vertices of said polygon; 

(c) dividing said polygon , by using said line 

25 segment included in said polygon as a unit, into a 

plurality of partial patterns relative to each division 
vertex where the directions of said line segments change 
by a larger angle than a preset angle, and adding a flag 
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1 to the coordinates representative of said division 
vertex <S4); 

(d) storing into a memory partial pattern 
collections for respective said recognition-object 

5 pattern and dictionary object pattern by performing 
the steps (a) to (c) ; 

(e) selecting a pair of partial patterns among 
the pairs derived one by one from each of said partial 
pattern collections, said selected pair having a 

10 smaller difference than a preset value between the values 
of each sum Jl^ and & 2 of l ine segments on the paired 
partial pattern and the values of each distance dl and 
d2 between start and end points of said paired partial 
pattern, and giving a coordinate transoannation matrix 

15 for the alignment of the position and orientation of 
one of said paired partial patterns with the other of 
said paired partial patterns (S51) ; 

(f) classifying said coordinate transformation 
matrix obtained for each partial pattern into the same 

20 group if the value of said matrix is within a preset 
error range, and giving an index f (j) which differs 
depending upon the presence .or absence of said partial 
pattern of said dictionary pattern (S52) ; and 

(g) giving an evaluation function MATCH « 

25 E W. • f (j) by adding for each group the multiplication 
j 3 

results of a preset weight given to each of said 

partial pattern of said dictionary pattern. 

8. A method according to claim 7, wherein said 
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1 polygonal approximation step (b) includes the steps 
of: giving in succession to each pixel a resultant 
vector (162) of a predetermined number of vectors each 
indicative of the distance and direction associated with 

5 adjacent two of said pixels; and setting as a vertex 
of said polygon a point where said successive resultant 
vector changes to a large extent excessing a preset 
range. 

9. A method according to claim 7, wherein said 

10 polygonal approximation step (b) includes the steps of: 
giving an angle a i between two straight lines passing 
through each of said pixels on said contour line and 
respective two pixels being- oppositely positioned 
apart by a certain number of pixels, from said each of 
15 said pixels; and setting as a vertex of said polygon 
a pixel where said angle a: is smaller than a preset 
angle. 

10. A partial pattern matching system for a 

dictionary pattern and a recognition-object pattern 

20 comprising: 

(a) binary-coding means input with an image 
information for giving a binary-coded image pattern by 
comparing each pixel with a predetermined voltage 

(10, 101, 102, 103); 
25 (b) polygonal approximation means for giving 

a coordinate string of apexes of a polygon composed of 
a plurality of successive straight line segments 
approximately representing a contour of said binary- 
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1 coded image pattern (104) ; 

(c) contour segmentation means for judging 
a corner point among the vertex coordinates of said 
polygon basing upon a criterion dependent upon the shape 

5 of said polygon , said corner point being vised for dividing 
said polygon into a plurality of partial patterns (106) j 

(d) means for providing a correspondence pair 
by subjecting each combination of said partial patterns 
of said dictionary and recognition-object patterns to 

10 matching comparison (107 , 109); and 

(e) means for evaluation of matching of each 
of said correspondence pair (108 , 110). 
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